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Gommunication plays an essential rple in disease 
prevention and health promotion! Recently, 
pi^ams. designed to promote changes in health be- 
haviors and to encourage early detection and prompt 
treatment of illness have demonstrated that mass ' 
media messages (1,2,5,7) can be effective in reducing 
risk of serious illness. ' 

Ck>nununicating'health and medical Information 
effectively, however, is a difficult venture. The sub- 
jects are-complex and often technical. In addition, 
the information to be communicated may be incon- 
clusive and subject to change as new research findings 
are released. Many diseases, such af cancer, are fear- 
arousing, and public responses are sometimes emo- 
tional. As a result, the potential is high for mis- 
directing or alienating target audiences with . 
inappropriate messages. 

Pretesting is important to assess comprehension, 
attitudes,'and other perceptions among the target 
audience in the early stages of communication 
planning and materials development. The diagnostic 
information learned from pretesting can lead to 
improvements in concepts, messages, and materials , 
while revisions are still possibfe and affordable. Prev 
testing will not guarantee success, but it can help 
reduce some of the uncertainty and risk of produc- 
ing materials that may be misunderstood or mis-' 
interpreted. 

During the past several years, the Office of Cancer 
Communications (OCC), National Cancer Institute, 
has used various techniques to pretest materials* with 
professional, patient, and public audiences.Tlanners 
in other health organizations also have pretested 
communication materials and have cooperated in 
sharing experiences and standardized approaches. 

The purpose of this booklet is to share these pre- 
testing experiences with health education planners 
and to encourage greater use of pretesting. The 
booklet describes the principles of pretesting, how 



pretesting can be incorporated into materials d^veN 
opment, and the most commonly used methods. It 
also discusses what resources are needed to conduct 
pretesting research. A glossary of pretesting terms 
and a bibliography can be found in the Appendix. 

The previous edition of this booklet was qalled ' 
Pretesting in Cancer CommAnications^ and'dealt * 
exclusively with cancer-related messages and mate- ' 
rials. Because of the widespread interest in the 
previous edition, and its application to other areas 
of health communication, thi$ updated booklet 
incorporates examples from otherhedth disciplines. 

Tw(rjother handbooks on pretesting are available 
from the Office of Cancer Communications. Read-, 
ability Testing in Cancer Communications provides 
step-by-step instructions for performitig a readability 
test and for improving the readability of draft 
materia^ The Health Message Testing Service describes 
a standardized approach for pret^ting radio and tele- 
vision-public ser^ce messages. 

To request additional copies of this booklet anc 
^ other information on pretesting and evaluation, ple^i 
contact: 

Rose Mary Romano 
Of f ice of Cancer Communications 
National Cancer Institute 
Building 31, Room 4B39 
Bethesda,MD 20205 
(301)496.6792 
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>ose 
of Pretesting 



What is Pretesting? . 

Pret^ting is one type of formative evaluation re- 
i^arch that is conducted in the early or forma- 
tive stages' of program development. The term pretest- 
ing IS used to describe the process of $ysteip^tically 
•gatherihgrtarget audience reactions to messages and 
materials before they ar6 produced in final forrn. 

Pretesting can help determine which of several 
alternative executions of an item may be most effec- 
tive. Or, it can identify strengths and weaknesses in 
single executions. The finding!^ can be used to revise 
N^nd improve ma);erials before they are distributed to 
sel6tted audiences. Mass media messages and materials 
that are suitable for pretesting include print items such 
jBiS pamphlets, booklets, posters, flyers, newspaper 
magazine advertisements, and audiovisual items such as 
television and radio public service announcements 
(PSAs), broadcast programs, films, and slide shows. 

What Does 
Pretesting Measure? . 

I^etests of messages and materials are usuaUy de- . 
signed to assess their effectiveness in the following 
categories: 

Attention -Does the message attract and/or hold the 
audience's attention? » 

Comprehension—Is the message clearly understooci? 
Are the main ideas conveyed? 

Personal Relevance— Does the target audience perceive 
the message to be personally relevant? 

• . V 



BelievabiUty~Is the nfessage and/^^ 
ceived as believable? ' 

Acceptability -Is there anything in the message that 
may be offensive or unacceptable tb the target 
^ludience? 

Additional gauges of message effectiveness may in- 
clude assessing target audience perceptions of the . 
utility of the information contained In an item and the 
extent to which target aud^^ find messages or ma-^ 
terials attractive, interesting, convincing, or alarming. 
Indjvidual production elements within an item also can 
be pretested. For example, the music in a radio , 
announcement, the cover art for a booklet, the chetr^c- 
ters in a film or television lanhouncement-all can be 
assessed. 

Pretesting in 

Health Communications 

Pretesting is an established technique in commercial 
marketing and advejctising. The essence of commercial, 
success is to provide prodifcts or services that meet 
copsumer needs. If the benefits offered are seen by 
consumers as unimportant, unnecessary, personally \ 
irrelevant, or otherwise unappealing or undesirable, the 
product or service may fail in, the marketplace; 

To avoid such failures; consumer perceptions, of , 
products and services are gathered prior to the full 
commitment of resources, before: 

• . considerable! dollai^ and time are committed ; 

• the risk is taken of alienating or misdirecting con- 
sumers; or 

• the point is reached when revisions are difficult or 
even impossible to make wi^^ 
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• Systematic pretesting has been adopted only re- 
cently by heialth education progran^ planners as an 

: integral step in the development 'of mass media. 

'messages and materials. Several factors have contrib- 
uted to the increased emphasis placed on pretesting. 
' Understanding characteristics .of the target audi- 
ence— its attitudes, beliefs^and behaviorsHs coniiidered 
an important elemient in producing effective messages^ 
Actually involving tai^get audiences in th^ formative 
stages of message development through pretesting can. 
ensure, a more effective comtnunipation e^c1iange« 
In the private sector, the effectiveness of a market- 

* ihjg program tp influence conlsumer.behavior usually 
can be quwtif ied in terms of increases or decreases 
in salei In heaW^ucation,it is more difficult to / 
aqhieve the desiredt)ril}ivior changes and to ipeasure 
program effects. The task of communicatiijg ef f ec- • 
tiyely also is mote difficult. Pretesting ensuresi;h^t 
mj^geb with th^ greatest potential to influence 
targeit audience attitudes and behavior are dissemi- 
nated. V 

Finally , pretesting in health education takes on 
greater importanpe when prograin resources are^arce; 
Agency administrators «i»fant to see results fron;i P^o- : 
gram expenditures. Although pretesting requires ex- 
, penditures of additional program doUfics^d may 
c^eatQ delays in the^evelopment of materials, these 
expenditures may minor compared to -total produc- 
tioli costs pf materials that are ineffective rbeteause they 
are misunderstood dr igre not accepted. 



Pretesting and 
the Health , 
Communication Process 



A fundamental premise of this booklet is that 
health communication programs must be based 
on an understanding of the needs and perceptions of 
their target audiences. The diagram belpw illustrates an 
approach to health communication that incorporates 
assessments of target audience needs and perceptions 
at critical points in program development and imple- 
'mentatidn. The six ^ages are a circular proces^ in 
which thi^ last stage feeds back to the first in a continu- 
ous process of replanning and improvement. 



The Office of Cancer Communications and other 
health agencies have used thLs'approach for developing 
and implementing numerous public and patient educa- 
tion program!;. It has served to organize communica- 
tion progratifs sb'that time and resources are aUocated 
efficiently for materials development as well as for 
pretestingand other evaluation research. « 

The six stages of health communication are de- 
scribed in this chapter to provide program planners 
w|th an understanding of this approach and, to show 



Figure l^ r 

Stages in Health 
Gommunication 




Shaded bcDtes indicate 
stages when pretesting 
is used. 
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how pretesting is us^d in the early stages of message 
development. The steps outlined below cdnstitute an 
ideal process, one that may require more time and 
q^oney than many agenctes can afford. All of the steps 
may not be feasible, or in some cases even essential. 
; Hence, program planners must apply their professional 
judgment as to what is appropriate for developing 
effective ij^essages and materials for their particular 
programs. 

... 'I 

Stage 1. 

Planmng and -..-^^^^^^^^^ _ 
Strategy Selection 

The health, communication cycle begins with defin- 
ing program objectives, identifying'target audiences, 
and establishing communication strategies that will 
provide consistency and cohesiveness to the program. 
To assist in this planning, inforhiafion on public 
knowledge, attitudes, and^havior is.gathered from 
€^xisting health surveys, state-of-the-art literature re- 
views, and results from clinical trials, v , 

The purpose of this research review is to find out 
. .what the public know^ believes, ^and dhes in regard 
to a particular "health subject. \Wiat misconceptions 
do people hold? Do they hav^ positive or negative 
attitudes toward certain health behaviors? Or are they 
ambivalent? What are the perceived l^enefits and 
.barriers to achieving behavior change? 

Research reviews may provide program planners 
with answers to these questions and also may provide 
clues for segmenting the target population into sub- 
groups. For example, national surveys on high blood 
pressure have indicated that the target audience of 
"aware hypertensives" (people who know they have 
high blood pressure) can be s'fegmented further into 
the following groups: V 

• those who take their medication regularly ; 

• those who tiake their medication occasionally 
(when they feel sick or nervous); 

those who have dropped pilt of treatment al^or,: 
gether. 



Often, communication planners find that available 
data are outdated, incbnclusive, or cannot provide 
enough direction for formulatiiig program objectives 
or for identifying audience segments. Hence, it may 
be necessary to conduct primary research with indi- 
viduals representative of the target audience. 

This research may be quantita±ive-i.e., conducting 
household surveys of a large, random sample ofre- 
spondents* It may be qualitative-i.e., interviewing a* 
small sample of respt&dents in depth. Or, it may fall 
somewhere in between-*i.e., using a combination of . 
qualitative research such as focus groups witli a small- 
scale telephone survey. Program budgets and time- 
tables will dictate what is feasible. ^ 

Whatever method is chosen, the purpose of this re- 
searcbshould be to find out, directly from people wlio 
are typical of th6 target audience, what they currently 
think and feel about a subject. The research should beV 
designed to bring to life all of the data culled from' 
research reviews and provide additional direction for 
program development. 

With this information in hand, program planners can 
f prmulate a communication strategy. The communica- 
tion strategy is a statement of the program |s objectives, 
the prinlary and secondary target audiences to be 
reached, and the benefits and other information that 
must be communicated. This strategy stiatement pro- 
vides writers and other creative staff with direction for 
all of the messages and materials that ar^ developed. It 
also may contain the tactics that will be.used to reach 
target audiences with the appropriate messages. 

If the communication strategy is to tell smokers 
how to quit, rttessages that simply exhort them to 
quit are "off strategy," Similarly, if the primary 
target audience for a breast cancer program is women 
over 40, communication channels must be selected 
primarily to reach these women. Channeling messages 
tojeach other age groups is a waste of program dollars. 
By carefully formulating the communication strategy 
and preparing promotion and distribution plans during 
Stage 1, time, money, and ener|y can be saved later 
ori: 



stage 2. 

Concept Develcipment 

On the basis of Stage 1 planning, message concepts 
are developed during Stage 2. These concepts are XiOt 
finished messages; rather, each contains an idea fot a ' 
full message. Concepts oftjen consist of rough art 
(a line-drawing or sketch) and litatements consisting of 
words or phrases to convey th^ main ideas for a 
. message. 

A series of 8 to 10 concepts^ for a television public 
service campaign, for instance^ niay involve different 
presenters (e.g., physicians, nursfes, or patients), varied 
sociological appeals (e.g., peer approval or disap- 
proval), or different approaches (e.g., emotional ox 
logical). Each concept, however, must be based fitrtily 
on the communication strategy selected in the plan^ 
ning stage. If the strategy centefs on communicating 
the effectiveneks of a disease detection procedure^ 
concepts that stress any other benefits or attribute 
are of f the mark. !"[ 

Pretesting at this stage can provide direction fox 
eliminating weaker approaches and identifying m^s^aga 
concepts that may have the i^ost potential. Sometirne^ 
entirely new concepts emerge from aijdi§jice response^. 
This winnowing process can save considerable time ^nd 
effort in the next stage when rough conj^epts will he 
used for crafting complete messages. 
_ For instance, in developing a mass media campaign 
u^to increase pujjlicrSwafeness of the health risks asSo^ 
ciated with exposure to asbestos, 4 message concepts 
were prepared and pretested. These concepts wefe 
based on a communication strategy which sought to 
incr€(^ase' public understanding of the problems of 
asbestos exposiire and also to convey the importance ' 
of taldjng certain actions if exposure has occurred- The 
message concepts were pi^tested smiong panel^of 
older, blue collar males and fenudes typical of me 
target audience. To give each message concept a visual 
dimension, a representative drawing was positioned 
next to several lines of copy and presented in poster 
form. Each concept used a different message presenter* 
These included an elderly shipyard worker, a doctor, 
the family members of a former shipyard worker, at^d 
a movie celebrity associated with World War H. Pretest- 
ing indicated that the message concept using a doctox 
.as presenter did not fare well with .male respondejits 
and was considered the least interesting visually. Itx 



addition, certain phrases used in the copV that ac- 
companied the other message concepts were found to 
Be misleading and were reyised in the message develop- 
ment stage. 

Another advantage to pretesting rough concepts is 
the gathering of words, phrases, and veriiacular from, 
thgj^target audience so that appropriate language can be 
used in formulating complete messages. For example, 
pathologists believed, a lay description of the symp- 
toms of melanoma (a form of skin cancer) should be 
"notched, blue-black, irregular spots." Interviews with 
members of the target population, however, suggested 
different adjectiVes entirely. When shown pictures of 
the symptoms, respondents used such terms as "looks 
like a bad sunburn," "a small rash," "blotchy," or "a 
bad scrape." 

Stage 3. 

Message Execution 

Once the concepts mth the most potential have . 
been selected, complete messages can be created in 
appropriate formats for reaching target audiences. \- 
These messages, produced in rough form, are pre- 
tested in the message execution stage. 

For instance, ra4K) announcements may be pro- 
duced in a nonstudio setting and with nonprofessional 
talent for pretesting purposes. A booklet can be pre- 
pared for pretesting, using typewritt/en copy and lay- 
- ing out rough artwork as it might appear ^ the final 
version. Similarly, posters, print adS; and flyers all 
can be mocked up in rough form for pretesting.^ 

^t the message execution stage, pretesting can be 
useful in the following ways: 

A. Assessing Comprehension 

Health messages and materials must be under- 
stood before they can be aptepted. For example, 
in pretesting a slide-tape presentation on breast 
cancer, program planners learned that the pre- 
sentation was considered generally cleiur and in- 
formative. However, the pretest also-revealed 
that the narration needed to be slewed down so 
that all of the information could pe better 
understood. 




> 
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Assessing Recall ; ♦ 

An essential inpedient of telkvisipn and radio 
lAs and posters is their ability to attract target 
audience attention. These messages are rarely 
seen or heard in an isolated media environment, 
and they must compete with other messages for 
viewer or listener attention. i% 

In pret€«5ting two different executions of 
television PSAs to promote running, program 
planners learned, that one execution achieved 
much higher levels of message recall than the 
other. The higher scoring message showed peo- 
ple, typical of the target audience, riinning and 
talking about their own personal exercise experi- 
ence. The lower scoring message'used special 
visual techniques and a voice-over announce^, an 
ex^ution which was not as effective In^^ttract- 
ing the attention of pretest respondrats. 




<^ Have you noticed all of a svdden 
every one4s running. They're manning 
to look better Running to feel better. 




i ', ^< / started to run because ray doctor 
suggested it. Now I run for me. 
Because I really enjoy it. It makes me 
feel sti^ng and more alive. 



D. 



Identifying Strong and Weak Points . , 

Pretesting prior to final production ^nd distri- 
bution 'can help ensure that each element of a 
message is likely to meet target audience infor- 
mation needs and perceptions. . 

For example, in pretesting a booklet on 
health risk appraisal, which contained an actual 
test for readers to complete, results indicated 
that the booklet was considered interesting and 
informative. However, the instructions for 
Iscoring the test were conAsing. Respondents 
needed^istance in figuring out their^ores and 
in plotting them on a scoring grid. 

. ■ 
Determining Personal Relevance 

. To have an impact^it is important that target 
audiences perceive that a message is pei^o|ia|ly 
relevant to them. . . 

In pretesting a-booklet on high blood pres- ~ 
sure among hypertensives and aniong ia general 
audience sample, results revealed several impor- 
tant differences in the responses of these two 
groups. Hypertensives recalled and understood 
more specific points related to high blood pres- 
sure control than did the general audience., 
group. Further, when asked whom the booklet 
was talking to, a higher proportion of hyperten- 
sives felt the booklet wa^ ''talking to someone 
li^e me." • ' 

. Gauging Sensitive or Controversial Elements 

Questions about audience sensitivity to sub- 
ject matter often arise in developing health 
messages. Pretesting can help in finding out 
whether materials may alienate or offend target 
audiences. | ^ 

For example, will a bare-breasted wjbman who 
demonstrates breast self-examination on tele- 
vision be an affront to adult viewers? Pretest 
results of such a message indicated that respon- 
dents held a range oT views. 



, .JO 
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While more than 8 of 10 
respondents felt the message 
performed a useful public 
service, the consensus was not 
as strong in regard to the issue • 
of the message's offensiveness, 
its suitability for a general ^ 
audience, or for airing any time 
of day Judgment must now be . 
applied as to whether the ' 
m^i^sage should be produced in 
its present form or revised. 



The Message 
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Performs a useful 
public service , > 

^ » ' ■ 

Is very educational 

Is suitable for airing any time 
of the day | 

Isn't at all offensive 

Is suitable for a 
general audience 



Percent 
Who Agree 

.84% 



Would have- been just as 
effective without showing a 
bare breast 



59% 
57% 



54% , 



c 

30% 
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In the remaining three stages of the health commun- 
ication process, programs are implemented and evalu- 
ated. Feedback from target audiences, however, should, 
not end with the pretesting that was conducted in the 
formative stages of program development. Based on 
informal observations, additional qualitative studies 
and process me^ures, program planners can make mid- 
course corrections to keep their efforts on track. 



Stage 4. , 

Implementation 

■ * 

FpUowing any revisions stemming from the message 
execution pretests, final production can be completed. 
The next step is to put the entire program into place in 
the field. At this point, care must be taken to monitcJr 
each aspect of the program. For instance^ the number . 
of phone ce^ or letters that have been generated by a 
mass media campaign can be logged. Or, monitoring 
servicesvjcanb? utilized to estimate the audience 
reach and the frequency of public exposure to televi- 
sion public service messages^ These types of informa- 
tion gathering on message and materials implementa^ 
tioh are known as process measures. 
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Assessing IrL-Market i - 
Effectivjeness 



The effects of a public education program or a 
• media campaign on public knowledge, attitudes, and 
behavior are measured during Stage 5 pf the health 
' communication process. For example^o assess the 
effects of a public awareness campaign on asbestos 
exposure, data abput public knowledge and beliefs 
were gathered by means of a national survey prior to 
the. distribution of mass media campaign materials, 
during the height^oTtte campaign, and again 3 months 
^ later. The point of this putcome evaluation is to assess 
program results; that is, what happened, in relation to 
original program objectives. 



Stage 6. t 

Eeedbac}^ to Stage 1 , 
Planning and ' ■ ; 
Strate^ Selection 

In tile sixth stage, all the informatioh gathered « 
through pret^tiiig and other' formative research, as 
well as process and butcome evaluation, is analyzed in 
preparation for a new cycle of program messages and 
materials. For example, if evaluation research indicates 
that messages failed to have the desired effects on cer- 
tain target audiences, this information can be applied 
in planning^new messages. The data should be ^ 
examined carefully to uncover problems an^ identify 
weaknesses that can b^e remedied in replannlng health 
communication programs. ^ , 
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troduction 



retesting encompasses various methods and ap- 
proaches. The methods that^are chosen depend \ 
on the nature of the materials, the target audiences, 
and the amount of time aiid resources available for 
pretesting. There are no set formulae for selecting the'' 
perfect method foi: pretesting. Methods must be 
chosen and shaped to meet each pretest need , with 
careful conisideration given to specific pretest objec- 
tives and the resources required. ^- C 

Many pretests in health communications are spe- 
cially designed to assess individual messages or ma|t%^-^^ 
rials. This process may be simplified, however, by " : 
drawing from methodologfes and questionnaires used 
by other agencies. The Health Message Testing Service 
(HMTS), describe later in this cha{)^er; is an example 
oi a standardized pretest method that uses the same— - 
core set of questions for each tesf^ To provide program"^ 
planners with direction for preparing a pretest ques- 
tionnaire, examples of st^daird HMTS questions are 
presented in the Appendix. V 

The following are examples of pretesting methods, 
adapted from marketing and communication research, 
which have been used by the Office of Cancer Com- 
munications and other health communicatibn * 
planners. ' 



r 
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Method 




Readability 
Ifesting* 




Acommoii criticism of bealtb ii^^ is that 

messages are not understood by the intended 
audience. Readability testing is a simple pretesting 
technique that can predict the reading icomprehension 
level a person must have in order to understand written 
materials. 

Readability formulae' use counts of language vari- ' ' 
; ables such as word and sentence length. The formulae 
have been devised statistically to predict comprehen- . 
sion scores. Actual participation By target audience 
respondents is not required. Depending upon the ' 
. formula that is chosen, a jreadability test can take as 
' little as 15 minutes to perform. 

Readability levels vary depending upon the length of 
the sentences and the number of polysyllabic words 
used. Generally, the longer the sentences and the ^ 
greater the number of polysyllabic words, the higher 
the reading level required. Because 
subjects tend to include many polysyllabic terms, 
program [Planners may consider including a glossary to 
define complex or technical terras. ^ 



Based on a review of the advantages,' disadvantages, 
and predictive validity of 12 selected readability 
f brmula^the Office of Cancer Ck>mmunications chose 
the I^OG Grading Formula for testing the readability 
levelsfof public and patie^it^education niKt«rials. 
SMOG was chose^n because it is one of the Amplest 
tests to use without sacrificing accuracy of prediction. 

Readability testing should be used as a first step in 
pretesting. draft manuscripts. Since it, does not indicate 
actual comprehension or other reactions, target 
.audience preteijBting should also be.cpnducted prior to 
final production of a booklet, brochure, or other 
printed message. , ^ 



1 



*A compxehemiye booklet on this B\xbiect, Readability Te$ting • 
in Cancer Communications^ is available free from the Office ^ Jf .^^ 

of Cancer CommunicationB. ' 



ErIco 



Readability 
Tfesting 



Summary 



Purpose 



Materials to be 
Pretested 



To determine reading gradeievel of draft manuscripts 



Leaflets, booklets, article, or other draft manuscripts 



Ideall^umber of 
Respondents 



No respondents-required 



Tiin6 Required 



15 minutes 



Resources Needed 



Advantages 



Readability fopnula 

Staff trained imeadability testing 



.Ine:dJ)ensive 
■ • Quick 

• Does i^ot 



Disadvantages 



provide target audience reactions 



1 tr- 
J. O 
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Method 



Focus 





V. 



Interviews 




Also called exploratory group sessions, focus group 
interviews are iised to obtain insijghts into target 
audience perceptions, beliefs, and language in the early 
stages of health communication development. Focus 
group intei^ews are conducted with a group*of about 
8 to 10 respondents simultaneously. Using a discussion 
outline, a moder^or keeps the session on traclc while 
allowing respondents to talk freely and spontaneously. 
As new topics related to the outline emerge, the ■ 
moderator probes further to gain useful insights. 
■„ Focus grptft[ interviews are a form of qualifotive re- 
search adapteai>y marketing researchers frotn group 
therapy, lliey are used for several purposes. 

In the planning stages of program development, v 
?fter reseai'ch reviews have been completed, focus 
groups can be used t6 develpp thie hypotheses (or ^ 
broad research issues) for larger quantitative studies. 
Kocus groups citn help determine public perceptions, 
misconcefptions^ and attitudes before a qu^onnaire 
is developed and the field research is cj^nd^ted..The 
areas of research can be probed in focus gi^wps to ' 
help generate ideas and develop hypotheses which . , 
wiU then be fully assessed in the larger, quantitative 
study. ^ ^ • , , 

For example, in planning a major national survey 
on public knowledge, attitudes, and practices related 
to breast cancer, researchers conducted separate focus 
groups^ with white, black, ^d Hisplaniq men and 
women to formulatfe the key issues. This'qual^tatiVe 
research was particularly imprortant with regard to the 



men and the minority groups be£ause, little informa- 
tion was available on their perceptions and beliefs 
regarding breast cancer. The focus groups helped re- 
searchers generate hypotheses and develop the word- 
ing for specific questions used in the national study. 

Focus groups also can be used to help interpret 
quantitative research. Marketing researchers originally - 
developed this technique to give them a, better under- 
standing of the data from large scale consumer surveys, 
^y obtaining in-deptii intonnation from individuals^ 
typical of the target audience, researchers can gain 
better insights into what the statistical data mean. 

Focus group interviews are especially useful as a 
pre*estihg tool in the cbncept development stage of the 
communication process. They provide insights into 
target audience beliefs on'a health issue, Idlow program 
planners to obtain perceptions of message concepts, 
and help trigger the creatiye^thinking of cominunica- ^ 
tion professionals, ihe group discussion stimulates . ^ 
respondents to talk freely, providing valuable^ lue^for 
developing materials in the consumers' 'own languagS. ' 

As with any qualitative research approach^ hQwever,* 
care must be taken not to interpret results quantita- 
tively. Hence, in pretesting message concepts, program 
names, logos, theme lines, or artwork, program P^^" - 
ners should look for the implications, suffiestions, or 
new directions that are indicated by this research; 

F6r example^ in pretesting potentid names and • 
logos for a county-wide heart disease prevention pro- 
gram in PennsylvMia, program planners conducted 
focus group interviews with respondents representa- 
tive of the county's population. Respondents' percep- 
tions of 10 piDSSible logos and program names, repre- - 
senting a range of ideas related to heart disease 
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prevention, were gathered. Preferences were exp 
for program names which specified the name of pe 
county. Names that contai|pe'd abbreviations wcifl re 
jected as confusing. In terms of the logo design; 
spondents preferred those which incorporated 
symbols of the program such as a heart or the 
of the county. These findings gave program pi 
direction for selecting a program name and cr 
a new logo design that incorporated both of t/j 
symbols. || 

As with all pretesting research, focus grou 
spondents should be typical of the intended 
audience. Various subgroups within the t^g 
should be represented. For example, in testii 
sage concepts on smoking aimed at a gener 
of smokers, a cross-section of individuals 
females, heavy and light smokers, older an 
might be recruited for the "focus groups. 

Respondients are recruited 1 to 3 weeks^ advance 
of the interview sessions, usually by telephcihe. They 
niay be recruited at random, using the tele/jhone 
- directe^, and interviewed to determine if 
for the group. Or, they may be recruited f 
of individuals representative of the target 
'Recruiting respondents at random is the p 
method. 

\^ There are several important criteria for 
effective group interviews. Respondents 
bjow the specifijc stibject of the session&^ advance, 
and they should not know each other. Mowing the 
subject may r^ult in respondents careMly formulat 
ing^ideas In advance and thus not talking spontane- ; 
ously about the topic during the sessi(fc. Knowing / 
other respondents'may inhibit indivi(|ii'als from'tallc- 




ey qualify 
m k list 
»pulation. 

fferre^^ 

!onducting j 
Ibuld not 



ing freely. Finally, 9II respondents sh/ 
comers" to focus group interviews, 
mor^ spontaneity in reactions and el 



Id be **jaew 



rJ 



I allows for| 
iidates the prob- 



lem of "professional" respo^idents who may lead or 
monopolize the discussion. ' \ 
/ It is desirable, Especially when pretesting on sensi- 
tive^ or emotional Objects, to segregate respondents 
oy age, -sex, race, or whatever other variable is likely 
to hinder freedom of expression. Teenage girls ard less 
likely to be inhibited in discussing Isexual activity, for 
' instance, if their parents, or teenage boys, are not in 
the group. 

There iis no set rule on the number of focus groups 
that should be condi^cted. The number of groups 
inevitably, depends upon program needs and resources. 
If target audience perceptions appear to be comparable 
from one group to the next, four focus groups should 
suffice. If perceptions vary, and the direction for mes- 
sage development is unclear, additional groiips may be 
necessary. 

An experienced, capable my)derator, 'who can skill- 
fully handle the group proceffi, should be used. The 



moderator must be well infi 
the purpose of the groups in 



rmed oh the. subject. and 
advance. A gQjDdjgradera- 



. tor builds rapport and trust and should probe respon- 
dents withoutreacting to and thereby influencing their 
opiiuons. The'^moderator must beable to /ead the 
disbussion, rather than be led by the" group: A good 
moderator keeps tlje discussion on tracK.while talldng 

. as little as possible and mV cl^ar that he or she is 
not an expert on the subject. 

. ^ . As noted earlier, the results of f o<;us group ^^fifer- 
yiews sh6uld be interpreted cariefully . It is usefuLior 
an unseen observer (behind a^one-way mirror) to .take 
notes as well as to^pe r^o/d the session for later 
revie^. In interpreting the findings from group inter-'^ 
vie-^, pro^^ planners sl^ould look for trends and . 
patterns in target audience perceptions rather than 
doing just a "he said . . . she said" Idnd of analysis. 

Focus ^oups should not be used when individual 
responses or quantitative information are needed. Foi: 
example,'wheri assessing the final copy for a booklet, 
it Is more important to gathier individual rather than 
group reactions. The former are mojre indicative of the 

^individual actual comprehension, perceptions, and 
intended^use. 



id 
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Focus Group 
Iritemews 



Sumn&y 



Purpose 


To obtain insights into targe^ audience perc^eptions, * 
beliefs, and language in the early stages of health com- 
munication development ^ \ 


Materials to be 
^ P^tested 


• TV, radio , or print messagf concepts, theme lines 

• Logos and rough artwork ^ \ 

■ ■ • ■ ■■• \ <^ 


Ideal. Number of 
Respondents 


• 8 to 12 per group \ 

• Minimum of 4 groups 


/Time Required. 


- • About 2 weeks for arranging groups and recruiting 
^ respondents 

• 90 minutes per group , " 

• 2 days to conduct groups 

' • 5 days to analyze interviews iand write report ^ 

• TotzJ time, from planning to completion of report, 
up to 3-4 weeks • ^ 


< ■■ • 


— — - : \ ' 7 : ; ■ ', 

• Discussion outline 

• Trained moderator . 

^ " •'^'Respondents typical of the target audience 

• Comfortable meeting room for conducting inter- 
* views 

^ ^ • Tape recorder * 
Onerway mirror for observing sessions (optional) 


A n V5^ ntj^ crps 

V. . • . . ^ . v» 

^ u.- _ . ^ . ' 


• ^Group atmosphere provides greyer stimulation 
* ' than individual interviews 

^ ^ 0 Excellent technique for obtaining qualitative in- 
formation from seyieral respondents at once 
f • Direction for message development stage can be 
gathered relatively quickly 


Disadvantages . 


^ . Should- not be us^d when quantitative data are 

< heeded foiydecision makingV^n message execution 
skgec^ 1 V ^ 
• Qualitativjejialaire of the research and small Sample 
V sizes do allow for developing norms against 
which to compare results 


> - ; . 
\v ■ ■ ' " , 
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In-Depth 
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I ndividual ib-depth interviews are used for pretesting 
issues ^at^must be probed deeply or wh^ individual 
rather than group responses are needed. Such inter- 
views can be quite lohg, lasting from 30 minutes to an 
hour, and are used to assess comptehension aS well as 
feeUngs, emotions, att^tides, and prejudices. Tliese are 
ar^ which are not normally elicited in, the more 
common public opinion interviews. 
. individual in-depth interviews, like focus group ses- 
sions, should be conducted by experienced interview- ^ 
ers who qsually follow a discussion outline. A sfhic- 
ture<if questionnaire can: be used in those casesVhere 
the pretester is concerned about obtaining reispon- 
dent$^ reactions to a core set olitems. 

The interviewer must be skilled at building rapport 
so that respondents feel comfortable ^d talk freely. 
Interviewers also should be adept at probing respon- 
dents about their feelings and reactions to the mate- 
rial$ bbing tested. The interviewerS sensitivity is 
crucial in conducting ihterviews«on lamotiohal subjecfo 
and'with individuals who may react emotionally to the 
materials being pretested. / 

bi-depth interviews can. he conducted nearly any- . 
where, althdugh a quiet spot where both interviewer 
and respondent can concentrate is preferable. Inter- 
views eoq[4iicted in the home mair be useful when the 
session refqiikes denionstration or easy recall of in- > 
home practices, or when specific, hard-to-reach indi-, 
viduals; such^as iober-city 'w6nien, niust be involved 
in"^ pretesl. With |iati0tit populations,'a .quiet office 
at the clil&ic or the hospital is an appropriate site for 
conductiro in-depth intenriews^ 
$eflBions alRrws the intiende^er to Concentrate fully 
oEf tike interaction and also tiftcj^itiatfes analysis of the 
interviews later on. Respondents should be assured of 
anonyioity and given an explanation of how the inter- 
pew rteulte wiU be useid. > 



» When pretesting booklets through in-depth inter- 
views, respondents can serye as "copy editors."* For 
example, in pretesting draft manuscripts of materials 
intended for teenage cancer patients, respondents were 
asked to read the drafts and to write their comments 
and suggestions directly on the drafts. Respondents 
were asked to underline those parts they felt were * 
most important, to cross out what they did not like, ^ 
and to put a question mark on parts they did not 
understand; After respondents had finished reading 
and commenting on the material, they were probed 
in depth about their reactions. . 

Because of the amount of time involved in conduct- 
ing in-depth interviews, it is advisable to schedule 
appointments in advance. For special target audiences,^ 

^ such as patijents or elderly people, staff members at 
clinics or sen|or citizen residences may assist program 
planners in identifying respondents. ^ 

Pret^ting through indMdual in-depth interviews 
iis often-applied at the concept and message develop- 
ment stages. like focus groups^in-depth interviews 
represent a qualitative method, usually conducted 
with smaU numbers of respondents. 'IRie subjective 
nature of responses and the small sample size require 
careful interpretation of results. The information 
gathered througl probing respondents in clepth is an 
aid to professional judgment, and should hot be used ; 
to make broad generalizations without further re- t 
search confirmation. ' > ^ - / 

Certain disadvantl^ of the in-depth interview ap- 
proach should be considered by program planners. The 
amount of time and expense ihvolved in.recntitiriie 
respondents, administering the interviews, and anal^z- 

^ ing the results^are the most significant factors. None- 
theless, the technique can be very useful for gainiing 
rich insights, especially when the subject matter is 
sensitive. , , 



r 



^ThU copy editing technique ckn be uaed in comMnation with 
other pretesting methods as well. ) 
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-Individual , 

In-Deptn 

Interviews 


. * , * 


Suriimaiy 


Purpose 


s 


In depth probing of target audience attitudes, be-^ 
lief and emotions 


Materials to be 
Pretested [ 




• Messale concepts ' 

• Draft mahusexipts, includtng those on sensitive or 
emotional subjects ^ ' 


Ideal^Number of 
Respondents 




Mipimum of 10 to 25 


Time Recftiired 


. . A' 


• 3 weeks to design questionnaire and arrange mter- . 
views ^ 

• 30 to 90 minutes per interview * 

• Number of days required* to conduct interviews 

' varies depepding upon availability of respondents 

• '5 to 10 d^s to analyze interviews and write reporlj 
' • Total time, fr&n planning to completion of report^ 

up^to 4^-Wek6r - / 


Resources Needed ■ 




• Discussion outline or questionn^re 
^ • Trained interviewer . 

• Quiet room • 

• Tape recorder 1 


Advantages 

if ' 

p. . ^~ . ^ ■ 


■ ■ ' ' / ' 


• Provides opportunity to prdl)e ind/vidud respon- 
dents in depth . f ' - 

^ Can obtain pretest information on sensitive or ^ 

• Good for jntervie\K|pg hard-to-reach audiences 


: Disadvantages ' - 




Time-consuming(:o arrange, conduct, and analyze 
/ ' r^ults . 
• The qualitative information obtained should not be 
used to make broad generalizations 




.r 
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Method 





Intercept 
Interviews 




Central location intercept interviews involve station- 
ing interviewers at a point frequented by individuals 
from desii^d target audiences and asking them to par- 
ticipate in the pretest . "There are two advantagei^ to 
this. First, a high traffic area can yield number of 
interviews in a reasonably short time'. For instance, a 
Baltimore shopping mall was used to interview 340 
men and women on DES (a drug'once used to ]^rev^nt 
miscarriages) messaged in 2 days. Second, a central 
location for hard-to^rekch target ajdiences can be a 
cost-effective means of g&thering data. In pretesting 
^skin cancer communication concepts,^ the centra^'^loca- 
I tions chosen were construction sites and l^each areas. 
These locations were eoccellent for locating r&pondents 
who are exposed excessively to the sun. 

A typical centrlil location interview begins with [the ^ 
intercept. Potential respondei^ are stopped and a^ked 
whether they would be willing to participate. T^en, 
spiBcific serening questions are asked to see whether 
they fit the criteria of tberlarget audience for the pre-. ' 
. test. If so, they are taken to the inte^ewing station^ \ 
quiet -spot at a shopping mall or othei site-tmd ai^e 
shown the pretest materials. Respondents then are 
asked a series of questions to assess recall, comprehen- 
sion, and reactions to the items. ^ 



Althougl]^ the respondents intercepted through cen- i 
tral location interviews may not be statistically repre- 
sentative of the entire target population, the sample is 
Isfgex than those used in focus groups 6r individual in- 
depth interviews.* Program planners often use the 
central location technique a^the message development 
stage when assessments of comprehension, attention, 
believability, and other reactions are essential. ' 

Unlike' focus groups or in-depth interviews, the ques- 
tionnaire used in^central loc^tiorr intercept pretesting is 
highly structured and contains primarily multiple 
choice or c iosed ended questions. Open ended ques- 
tioning, ^lich allows for jfree flowing answers, ^ould 
be kept to a minimum because it takes too'much time 
for the intc n^ewer to record responses. The question- 
. naire, as in any type of research, should be pilpt tested 
'before it is used in the field. ' 

A numbcjr of marketing research companies through- 
out the country conduct central location intercept 
interviews in shopping^aUs. Clinic waiting rooms, 
churches,'Social Security offices, schools, or other 
y locations frequented by individuals representative of 
\ the target audience also can be used for this purpose. It 
i| advisable to obtain clearances or permission to set up 
V interviewing stations in these locations well in advance. 

>n pretesting a bilingual (Spanish^nglish) booklet^ 
on breast feeding for a pilot nmtemal and child health 
care program, a marketing research company with ' 
{ bilingual interviewers was used to conduct pretest 
interviews at.severial clinics in a large metropo^tan 
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. ...area: These clinics served members of the target audi- 
^ ence of pregnant women and women who had recently 
given birth. 

. During the pretest, interviewers were stationed in 
thie clinic waiting rooms to intercept respondents who 
then were led to a quiet location for the actu^ 'inter- 
view. Respondents were given jthe breast feedingHiook- 
let to read. When they were through, the. interviewers 
returned to administer the pretest Ijuestidnnanres The 
total amount of time required, from interipept to " 
completion of the interview, avei'aged|ib6ut 35 
minutes. Using six interviewers, two at each of three 
sites, 150 interviews were completed in about 3 days. 

The major advantage of thd central location mter- 
cept approach is its cost-effectiveness for interviewing 
l^ge numbers of respondents in a^short amount of 
time. Designing a central location intercept pretest * 
; also can be relatively easy. Gtee or two screening 
questions, to identify respondents typical of the de- 
sired target audience, should be administered at the 
point of intercept. Questions to assess cbmprehensioii 
and target audieiice perceptions of the pretest ihate- - 
rials form thecore olythe questionnaire. A few addi- 
tional questions, tailpried to the sjpecific item being" 
tested, also may be constructed to meet program ^ 
planners' particular needs. 

Central location ^tercept interviews should not be 
used if respondents must be interviewed in depth on 
emotional pr v^ sensitive subjects. The intercept 
approach also may not be suitable if respondents arc^ 
likely to be skeptical or resistant to being intercepted 
^ on the spot. Althcnigh it isjiiore time-consuming to 
set up prearranged appointments, this approach may 
save time in the end if resjpfondents arp not willing tc 
<vcooperate. 



Central Loeation * ■ 
Ijitercept Interviews 
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Summary 



■f 



Purpose 



Materials to be 
Pretested 



Ideal Number of 
Respondents 

Time Required^ 



Resources Needed 



Advantages 



Disadvantages ^ 




5 



To obtaiil target audience reactions to concepts and 
messages from fairly large numbers of respondents in a 
short period of time 

• Message concepts ' , 

• Prin^matericds such aS booklets, leaflets, posters, ads 

• Broadcast m^lssages such as radio and television 
PSAs . ^ 

150to200~ ^ . ■ .1 , '~ 



3 weeks to design questipnnaire and arrange inter- 
views - \ - ; - 
Length of interview depends on pretest design; 
average is about 20 to 30 irimutes per interview 
Number of days required for' field work-varies ^ 
depending upon iengtb of interview, number of 
interviewers, and traffic in ^entral location{ average 
is about 4 days ' - . 'V 
10 days to tabulate resujte and write report 
To||al time, from planning to completion of re- 
port, is up to 4-6 week^ 



• "Structured questionnaire 

• Trained Jnterviewers 

• Access to central location frequented by indi- 



viduals ty^ri^dlDf target a^ 
• Interviewing station 



Quick method for obtaining large numbers of 
interviews ^ 

• Flexible^ for pretestiiig many types of materials 

• Technique can be adapted for pretesting in a variety 
of locations - 

• Use of many closed ended questions allows for 
quick analysis of results 

• Inappropriate for probing on sensitive or emotional 
subjects 

• Interviews cannot be long ^ 



2 3 



19 



EKLC 




Method geif- 

5 Administered 
Questionnaires 



Self -administered questionnaires also can be used to 
pretest materials. These qucistionnaires may be 
mailed to respondents along with the pretest materials, 
or distributed to respondents gathered at a Antral loc- 
ation. The Office of Ctocer Communications has used 
this technique in cases wkerepersonal interviews have 
not been-feasible. Self-administered questionnaires can 
be an inexpensive prieitesting technique fo^gencies 
with minimal r^ources. . 

This tecnnique involves designifag a questionnaute to 
lifeet pretest phjectives, th^ pilot testing it with 5 to 
10 respondents. Usually, questionnaires and pretest 
materials are distributed to respondents whose partici- 
pation was sought in advance. Respondents are asked 
to review the ihaterials on their own/to cbnH>letelhe 
questioonaure, andlhen to return it within a specified . 

Hie questionnaire shoidd be a reasonable length or 
respondents may not complete it. Open ended quesk 
tions may be used to assess comprehension and overall 
tedctionsj to m^iterials and closed ended questions to 
assess such factors as personal relevance and believabil- 
ity of the material. Measures of attention or recall 
cannot be used with this technique tince there is no 
way of controlling respondents' exposure to the 
material. 

In a pretest of a long booklet on coping with cancer, 
the cooperation of several cancer patient groups and 
comprehensive cancer centers was sought. The booklet 
and. the pretest questionnaires were mailed to r(^pon- 
dehts by contacts at e^ch organization* These items * 
then were retumikl directly to/program planners after 
respondents had completed;, their questionnaire^. Be- 
cause respondents had marked the booklets with 
editorial comments, it was essential that these be re- 



I 
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turned as ivell. Self 4iddressed, stamped envelopes 
were provided jCor this purpose. To boost the response 
rate, follow-up telephone calls were made. / 

Self ^administered questionnaireilaUbw pro-am 
planners to eficudetailedlnfonQQtioii from respon- ^- 
dents who^ay|;iot be accessible for personal inter- 
views, such as doctors, teachers; or media profes- 
sional. They also allow rest)ondent|4:o maintain their 
anonymity, pother advantage of tl& method is that 
it does not require any interviewer time andldan be 
done relatively inexpensively. Resources are invested > 
in questionnaire developmei):t and analysis of resultSt 
The latter expend can be kept to a minimum by nising * 

. mUny clCMsed ended questiohs. ' . r ^ 

. : l^lf -administered questipnnaires^have ce^aih dis- - 

^advantages. The primary problem wi|^ this technique 
is the possibility of a low response rate and the need to 
recontact respondents to encourage them to retunv. * 
their questionnaires. It is important to over-recruit re- 
spondents to ensure a sufficient number olg returns* 
This method may take much longer than central loca- 
tion intercept interviews because of delays in 
responses.' 

Another problem is that the type of respondents 
who return the questionnaires may be different front 
those who did not respond. Hence, a certain degree of 
bias may be introduced and results should be 
interpreted with this in mind. 

^ . •. ■. ^ . - ' 
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Self-Administered 
Questionnaires 



Summary 



r 



Purpose 




To obtain target audience reactions to draft materials 


Materials to be^ 
Pretested ' ' 




Print ^niateri^ such as booklets or leaflets 

;: ■ ■ ^ , . •'" V 


Ideal Number of 
Respondents 




' Minii^m of 20 

y . ■ ■ •■ • - 

' ■■• . ■. 1 '-^ ^ ; - . 


IMe itaquired 


. » 
»« ■ 


• 2 to 3 weSks for designing questionnaire and r^- 
/ cruiting respondents. J • . 

' . • o CO 4 weeKs lor oDtainuig^re^ons6|p 

• 5 to 10 days for analyzing results and writing report 

• Total time required from planning to completion ' 
of final report is up to 6 to 8 weeks ' 

, — V '—- ; 4- ^ 


Resources Needed 




• List of potential respondents ^ . 

• Structured questionnaire; 


Advantages 




5 1 ~ . 

• Inexpensive 

• Does not require time for interviewing 

• Respondents maintain anonymity 

• Can reach target audiencesinaccessible through 
central location intercepts or thdse unlikely to 
cooperate with personal interviews 



; Disadvantages 



Response rate may be low and cause delays 
Respondents are self -sel^ted, introducing a certain 
degree of bias 

Respondents' exposure to pretest materials cannot^ 
be controlled 
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T hie Office of Cancer Commttnications has joined 
with ^the National High Blood hriessure Ediucatipn 
Program, the Office o^Health Information and Health 
Prdnjotion, and the Office of Public Affairs of the 
UJS. JDepartment of ^Health wd Human Sem^ 
sponsor a standard system of pretesting health public 
servio^ announcements for radio and television; IjD^T^ 
testing can help message prddttC^ assess their mes- 
;sa|(^ priQC tor finid production and gauge the! folio W- 

ii> the abiUty of a messiige^to attract attention^^ 

• recall of important ideasV : ■ V 

.• whether a message is considered bf the target audi- 

.. ence to be personally relevant, believable, interest* ■ : 
♦ ing, and ihf otniative; and 

• other indicatioiis of message strengths and weak- 



nesses. 



The seryic^ Is currently provided free of qharge to 
health and human service organizations. The following ' 
is a bri6f description of how H^ 

Health message producers submit a prefinished lY ^ 
or radio spot, as well as information on message ob- ^ 
jectives and target audiences, ^6 HI^&s^ x: 
Office of Cancer Communications. Testing is con- 
ducted with 300 respondents in three geographically 
diverse cities. Ihe respondents are recruited by mail. 

At each test location, respondents view a pilot TV 
show br listen to a pilot jadio program. Within / 



^ show are both t^t and control messages including the; 
HMTS mes^g^ an^ 

services. Responded or hear each tSt messsage : 

Following eaqh exposurelo thejme&age, attejp^oni 
and main idea communication are measuried by asking 
the audience tg indicate all the messages'they ren^m- • 
' ber (iittention) as well as the main: idea of each ; ^ 

A series bf standard questions usied in all HI^TS test- 
ing thjen is administeiied to measure cei^n audience 
Responses such as diBgree of )>(eUevability, ^ersonlal tele- 
vaiifce, and comprehension. Tliese standcqrctlneasures ' 
alsovkre used for compiEurm 

tested spots. Finally^ a series of questions, st>ecificaliy 
designed for each message test, is administered to . 
assess audience comprehension of message copy points 
and to address any sensitive, controversial, or question- 
able elements in the n^essage. >^ 

A final analysis and report of the test results is 
available about 5 weeks after the test date* Pifeliminairy 
results are provided in about 2 weeks: 
. The Health Message Testing Seirvice has been ex- 
panded to include print messages and materials. ' ^ 
Health-related institutions and other organizations may 
contact the Office of Cancer Communications to 6b- 
y^ain further information about the service. , 



*The Health Message Testihg Sewke: m Standardized Approach 
for Assessing Audience Response to Health Messages is another 
booklet available from the OfHce of Cancer Cotnmuhications 
viiilch describes HMTS in Retail. 



ERIC 



Health Message . . 
TMting Service (HMTS) 




Purpose ' ^ 




■ . . . • • • 

' To assess attention, comprehension, believability, ' 
and other target audiencejreactions with fairly large 
samples of respondents . 


Materials to be 
^Pretested^ 




• Radio and television PSAs 
; • Printed materials such as booklets, posters, flyers, 

' .cuiu alio 


Ideal Number of 
Respondents 




Minimum of 300 

■ .."»•.■ ' . ' 


TiTTfip R-POTiirpd : 

• * . . * ." . • . 




• 2 weeks for designing questionnaire and identifyii^ 
riespondents ^ 

• 2 days for conducting field work 

• 2 weeks for tabulating results . 

• 1 to 2 weeks for analyzing results and writing 
report 

' • Total timis required, from submitting message to 
. , • - completion of report, is up to 8 weeks; preliminary 
" ^ data ar^ available in 2 weeks 


Resources Needed L 




• Rough produced PSA or print item 
, . Statement of message objectives and target 
. ' ./ audience 

HMTS supplies questionnaire, data analysis, and 
report 


' .. . . J . . . * J ■ f ■» 

Advantages 

' ■ . ■ ■ . ' . ■ 'V 

.-■ >.■ i" * " ." ■ • • ■ ■ , • , ■ * * 
•_ a 1: J_i — '. — -J ■■ — : '■ '■ 




■ ' ; • Standardized approach 

• Results can be compared with results of previ- 
ously tested messages 

• Large sample size in 3 geographically diverse 
locations 

/ / • Provides results on basic pretesting measures and 
can tailor questions for specific messages 



Disadvantage^ 



Requires up to 8 weeks for results . 
Limit on the number of open ended questions that 
can be asked 



Method 





er 





O ften, public and patient education materials are 
distributed to their intended target audiences 
through health professionals or other intermediary 
organizations. Ihese intermediaries act as aaitekeepefs, 
controlling the distribution channels for reaching target 
audiences. Their approval or disapproval of materials 
is a critical factor in ia program's success. If they do 
^ not like a poster or a booklet » it may never reach the 
public. 

Although not a pretesting technique in the strictest 
sense of the term, gatekeeper review of rough materials 
is important and shoidd be considered part of the 
fonnative evaluation process. It is not a substitute for 
pretesting materials with target audience representa- ^ 
tives. Neither is it a substitute for obtaining clearances 
or expert review for technical accuracy; these shoidd 
I^e completed within the organization before pretest- 
ing is undertaken. 

Gatekeeper reviews are conducted simidtaneously 
with target audience pret^ting so that data from both 
groups can be gathered, analyzed, and synthesized 
to provide direction for revising materials. A short> 
self -administered questionnaire, directed to individ- 
uals representative of the gatekeeper population, is a 



sufficient tool for conducting this review. Questions 
may include such areas as overall reactions to the 
materials and assessments of the appropriateness, 
completeness, and utility of the information. v 
For exampte, while cancer patients and family mem- 
bers reviewed the coping with cancer manuscript de- 
scribed earlbsr, copies also were sent to staff at the 
Cancer Information Service (CIS), a toll-free tele* 
phone network that provides information about 
cancer to the public and to health professionals^ Be- 
cause the CIS offices woxdd be a key distribution 
channel for the booklet, staff comments were solicited 
during the message executi||iii stage, and considered 
along with target audience responses in formidating 
recommendations for revising the booklet. 
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Gatekeeper 
Review 




Purpose 



To gather reactions of potential gatekeepers to draft 
materials prior to final production 



Materials to be 
Pretested - 



Printed or audio-visual program materials 



Ideal Number 
of Respondents 



Minimum of 10 to 25 



Tiiri 


le Required 


• 3 to 4 weeks to obtain responses through self- 
administered questionnaires,' 






• 1 to 2 weeks to analyze resiilts 






• Up to 4-6 weeks from planpmg to completion of 






report 



Resources Needed 



List of potential respondents 
Short, structured questionnaire 



Advantages 



Inexpensive 

Provides direction for revisions from a critical 
population 



Disadvantages 



May get low response rate 

Results based on small sample and should be 

interpreted carefully 



r 
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inclusion 



■ ~ -~ ^ 

Pretesting methods should be selected dnd designed 
to meet the specific needs and objectives of pro- 
gram planners. Focus group and individual in-depth inter- 
views are useful qualitative research techniques during 
the planning and concept development stages. Read- 
ability testing should be used as a first ste^ln pretest- 
ing draft manuscripts, then followed by pretesting with 
target audience respondents. CSentral location inter- ^ | 
views with larger samples of respon^lents, or pretesting ^ 
coniiparable to the type used by the Health Message 
Testing Service, should be applied during the message 
development stage. Self -administered questionnaires 
for pretesting printed materials with target audiences 
or for conducting gatekeeper review are inexpensive 
pretesting metholtis. 

Figure 2 on page 27 presents the six stages of the 
health communication process and the various pretest- 
ing and evaluation methods that can be applied at each 
stage. ^ 
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Fig^ure 2 



Pretesting and Research 

Methods Applied in the 

Six Stages in Health Communication 



J 




1. PLANNING & ' 
STRATEGY SELECTION 

• Review state of the art 

• Analyze available health studies 

• Determine target audience " 

• Supplement available data with 
baseline studies: 




6. FEEDBACK TO PLANNING 
& STRATEGY SELECTION 

• Analyze all data collected from: 

baseline studies 

concept testing 

message testing 

assessing audience reach and 
exposure to messages 

compiling process measures 

• Interpret data and synthesize for 
replanning new cycle ot messages 
and materials 



small scale surveys 
focus groups ! 
in-depth interviews 



2. CONCEPT DEVELOPMENT 

• Focus group interviews 

• Central location intercept interviews 

• In-depth interviews 



3. MESSAGE EXECUTION 

• Readability test print materials and 
scripts (in-house) 

» -Conduct message testing via: 

Health Message Tbsting Service 

Central location intercept 
interviews * 
Self-administered qtlestionaires 



5. ASSESSING ■ 

EFFECTIVENESS 

Assess impact through studies of - 
awarene^ attitudes, and behavior, 
using: 

telephone surveys 

large scale omnibus siarveys 

Compile process measures of media 
and materials usage through: 

survieys of media gatekeepers 

review of station logs 

inquiries to community- 
oi^gifmizatiori lead^l^ 



4. IMPLEMENTATION 
Final production 

Distribution -r 

Assess audience reach and exposure 
to health messages and materials 
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Limitations 
of Pretesting 



Pretesting can provide program planners with ob- 
jective information and direction for improving 
concepts, messages, and materials while revisions still 
are possible ^d affordable . However, given the qualita- 
tive nature of mixst pretesting research, it is important 
to recognize certainlimit^tjions. > 

Pretesting is neither an absolute pK^dictor nor a 
guarantee of success in terms of leamingypersuasionj 
t^ehdvior change^ or other measures of communication 
effectiveness. 

Pretesting in health communication is seldom de^ 
signed to quantitatively measure small differences ^ 
among large samples; it is not statistically precise. It 
will not reveal that ExecutioB A is 2.5 percent better 
than Ei^ecution B. Presumably, pretests of such pre- 
cision could be applied, but the cost of obtaining such 
data would be high, and the findings may be no more 
useful than the diagnostic information from more 
affordable approach^. 
V Pretesting is not a substitute for experienced judg- 
ment* Rather, it is a tool to provide direction from 
which sound decisions can be made. 

Given these limitations, it is important to avoid 
misuse of pretest results. Perhaps the most common 
error is to overgeneralize. Qualitative, diagnostic pre- 
test inethods should not be used to estimate broad- 
scale results. K 5 of the 10 respondents in a focus • 
group interview do not understand portions of a 
pamphlet, it does not necessarily mean that 50 per- 
cent of the total target population will be con- 
fused. This lack of understanding among the pretest 
respondents suggests, however, that the pamphlet .may 
need revisions. to improve comprehension. In sum, pre- 
testing is indicative, not predictive. v 

Another problem that arises in health communica- 
tion pretesting concerns interpretation of respondent 



reactions to sensitive or emotional subject matter such 
as breast cancer. Respondents may become unusually^ 
rational when reacting to such pretest materials, and 
cover up.their true concerns, feelings, and behavior. 
As a result, the pretester must examine and interpret . 
responses carefully. » 

Pretesting offers both the opportunity and the 
temptation to structure the test and interpret results 
to support or justify a preconceived point of view. 
It is natural to want favorite concepts or messages to ^ 
test well, but it does no good to prove a point in pre- 
testing only to have it disproved after a full commit- 
ment of resources. 

One final point: pretesting does not necessarily 
lead to flawless execution. Good planning and sound 
pretesting can be negated by mistakes in final produc- 
tion. The message in a television PSA on cancer 
treatment, for instance, may pretest well, but then be 
flawed by an execution that uses an actress who seems 
too happy to be awaiting the results of a biopsy report. 
Similarly, leaflet copy that pretests well may be 
rendered ineffective by a poor layout, hard-to-read 
type, and inappropriate illustrations. 
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Planning 

and Conducting 

Pretesting Research 

1 ' ^ 



Designing the 
\ Questionnaire 



The level of effort and staff resources required for 
pretesting will vary from one pretest to the next. 
While it would be^deal to pretest messagjes and mate- 
ti^s with large samples of representative, randomly 
recruited respondents, this is not feasible for most 
organizations. Instead, most pretesting is conducted 
with small saniples consisting of respondents who are 
typical of the target audience and who are easily ac- 
cessible for pretesting. This approach imposes certain 
limitations on generalizing about pretesting results. ^ 
However, these results, combined with professional 
judgment, provide important direction for improving 
Qiessages and materials prior to final production. 

^The following discussion is intended to provide pro- 
gram planners with practical sugg^tions for how to 
plan and implement pretesting resiearch. These sug- 
gestions should help program planners reduce the 
time and costs involvi^ in pretesting research, ' 
whether or not commerical research firms are hired 
to supply field work and tabulation services. Rough 
cost estimates for cdnducting pretesting research using 
commercial research f inns are presented in the Ap- 
pend| x. Direct costs are estimated for the major pre- 
testing methods described earlier. 



As in the planning stage of program development, a 
first step in planning a pretest is to formulate the re- 
search objectives. These objectives should be stated 
specifically to provide a clear understanding of what 
program planners want to learn. Measures of attention, 
comprehension, believability, and personal relevance 
are the key communication pretesting measures. Oth^r 
' questions to identify strengths and' weaknesses in 

rough messages and materials also should be developed 
, based on the pretest objectives. Ques^'o/is sAou/d not^ 
be asked just to satisfy someone 's curiosity. 

There are several ways to keep pretesting costs 
down. First, keep the questionnaire short and tathe 
^oint. Second, for easy tabulation and analysis, iry to 
use as many closed ended or multiple choice questions 
as possible. When using open ended questions, try to 
develop the codes for quantifying these responses in 
advance. For example, wl^n asking an open ended 
question on the main ideas a booklet is trying to get 
across, list the main point codies on the questionnaire. 
That way, free flowing responses can be categorized 
and tabulated relatively quickly and accurately. 

Whenever possible, borrow questions from other 
pretesting research. The examples from HMTS testing 
that appear in the ^pendix are one source. The 
reports available from the Office of Cancer Comfhuni- 
cations and listed in the Appendix are another. Com-, 
muhications Pretesting^ listed in the Bibliography, 
contains descriptions of methodologies and question- 
naire for testing radio and television announcements, 
posters, pamphlets, and films. " 




f 



Recruiting Resporidents 

If program budgets do not aDow^for recruiting re- 
^spondents through randomized technic|ues orihrough 
research^ firms, program planners have a number of 
optioit^Tb^cniiting Respondents typical of the target 
audience. ^ 

For focufi group or individual in^lepth interviews, 
seek the cooperation of local church, civic, social, or 
fraternal groups who have an interest in the materials 
or programs that are being developed. These groups 
niay be willing to assist in the pretest. A small dona- 
tion in return for their help is appropriate. For focus 
group interviews, it is best to recruit respondents who 
do not know each other. Hence, tl^ membership lists 
of several local' groups should be used. 

To pretest materials targeted to patient or special- 
ized health audiences, contact clinics, hospitals, or 
HMOs in your service area. Clinics may be an excellent 
place for conducting in-depth or central location. inter- 
dept interviews. Hiey also may be used to recruit a 
pool of respondents for a pretest using self -adminis- 
tered questionnaires. 



Recruiting Interviewee 

Trained interviewers should be used in pretesting re- 
search whenever possible. For focus group and in- 
depth interviews, this is essential. Agencies that have 
no experience in focus group research might consider 
, starting by hiring a good, experienced moderator, ob<* 
serving and taping the sessions, and using them as train- 
ing for developing in-house skms. Ix>cal advertising 
agencies may be of assistance in identifying a good 
moderator. Continuing education courses in inter- 
personal communication or group interaction also may 
be useful. * 



For conducting central location interviews, univer- 
sity and college departments of marketing, communi- 
cations, or health education might be helpful. Pretest- 
ing a poster or a^' advertisement is an excellent "real 
wprld^ research project which faculty members might 
be interested in adopting as a class project. Students 
in these, departments are being trained in research 
methods,and pretesting would give them the chance 
to develop then: skills. . 



Facilities 




Pretesting facilities should be quiet and comfort- 
able. Meeting rooms at churches, office buildings, or 
other institutions can be used for conducting focus 
group or individual in-depth interviews. If an observa- 
tion room with a one-way mirror is not available, 
another means of allowing staff memt^rs to observe 
the group is to hook up speakers in a rdlj^ nearby. 
That way, staff can listen to the interviews as^th^y are 
happening. , * 

For central location intercept interviews, it is 
important to obtain permission for conducting the 
interviews in advandl. The interview sta^on need not 
b6 any fancier than a bridge table 'and two cljairs. It ^ 
should be situated away from highly traffickra areas 
to avoid distractions. / 



Obtaining Research 
Assistance 

Many resources exist for obtaining prof&sional as- 
sistance in pretesting. The faculty at university depart- 
ments of marketing, communications, health educa- 
tion, psychology, or sociology can be helpful in i 
designing and conducting pretests. 

Many communities have marketing research firms 
that specialize in respondent recruitment, interviewing, 
tabulation, an^d other services. Such firms also may 
have facilities for conciucting group sessions and other 
techniques. The Greenbpok: Intepiational Di^^ 
of Marketing Research-Houses and Services identifies 
suppliers and services for each^^y throughout the 
United States. Also, there are adv^tising clubs 
throughoi:M;,the country, many affiliated with the 
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American Advertising Federation, that undertake 
public service communication projects, sometimes at 
no charge to nonprofit organizations. \ 

Individuals trained in commercial testing may not 
be completely aware of all the nuances and subtleties 
involved in health communication. They should be 
able to draw on their advertising research experience 
for selecting the appropriate pretesting methodology. 
But, there are many factors, such as primary and 
secondary audience selection, concept and message 
development, sensitivity of subject matter, and 
wording and interpretation of questions and results 
that are Influenced by the complexities of health 
information. The old adage that managers should ^ 
know enough about each facet of their business to 
manage their experts holds true for pretesting. Ex- 
perts can be used, but they shoi^M be closely super- ^ 
vised and guided. 

' In addition to these resourceis, the Office of Cancer 
Ciommunications can provide limited pretesting assist- 
ance to health agencies and organizations on request. 
Staff consultation for developing, conducting, and 
interpreting pretest results is available, as is the Health 
Message Testing Service for pretesting broadcast PSAs 
and printed materials. , ' 



Gonclusion ^ 

This booklet has been developed to provide progr£|flfi 
planners with an understanding of pretesting in health 
communications. To yield useful results, however, a 
pretest should be planned carefully. Ample time 
should be allowed f or : 

• contracting outside research firms (if necessary); 

• arranging for the required fapilities; 

• developing and testihg the questionnaire before field 
application; 

• recruiting interviewers and respondents; 
•sk gathering the data; v 

• analyzing the results; and 

• making the appropriate recommendations. 

' ■ ' - • ■ ■ i- 

Without adequate planning, pretesting*may not serve 
its intended purpose— to improve messages ai^d mate- . 
rials prior to final production. Instead, it may be seeq 
as an unnecessary expense for research that is of little 
or no use. The Office of Cancer Communications has 
not found this to be the case. For 6cCand many 
other health agencies, pretesting has been a valuable 
tool for developing commudlication programs that have 
contributed to improving the health of the public. 
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Appendi^c^^A 



Standard QuestioFrs Used by the 
Health Message Ifesting Service 



In pretesting radio and television public service 
announcements or printed materials such as 
booklets, the Health Message Testing Service uses a 
standard set of core questions to assess main id^a com- 
muAication, believability^personal relevance, and 
other target audience reactions. These stand^d ques- 
tions are listed below to assist program planners in 
developing pretest questionnaire^. The questions can 
be modified by changing the words in parentheses to 
fit the particular item that is being pretested. 

1. Main Idea Communication/Comprehension 
What was the main idea this (message) was 
trying to get across to you? 



In your opinion, was there anything in the 
(message) that was confusing? » 



Which of these phrases best describes the 
(message)? : 

. ' Easy to understand ; 
Hard to understand 



Likes/Dislikes * 

In your opinion, was there anything in particu- 
lar that was wov^h remembering ^libout the - 
<message)? . 



What does this (message) ask you to do? 



What action, if any, is the (message) recom- 
mending that peoplfe take? (Probe: What other 
actions?) 



W^ai, if anything, did ypu^particulatly like 
about the (message)? 



:Was there anything in the (message) that you 
particularly disliked or that bothered you? If 
yes> what? 
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Believability 

In your ppinion, was there anything in the (mes- 
sage) that was hard to believe? U yes, what? 



Which of these words or phrases best describes 
how you feel about the (message)? 

Believable 

Not believable 



4. Personal Relevance/Interest 

In your dpinion, what type of person was this 
(message) talking to? 

Was it talking to . 

Someone like me ' : 

Someone else, not me 



y«s it talldng to ... 
All people 

All people, but especially 

(the target audience) 
Only (the target audience) 



Wfliich of these words or phrases best describes 
how yoii feel about the (mes^ge)? 

Iiiterestingf^^^^^^^^^ ^ 
v Not interesting ' ^* ^ • • ; : 



Informative 
Not informative 



Did you learn anything ite^^bout (healtl^ sub;* 
ject) from this (message)? It y^es, \^^hat? 



5. Oth^r Target Audience Reactions 

Target audience reactions to pretest materials 
, can be assessed using pairs of words or phrases 
or using a 5-point scale. The following is an ex- 
ample of how this is done. 

Listed on this sheet of paper are several pairs of 
words or phrases with the numbers 1 to 5 be- 
tween them, rd like ypu to indicate which . 
number best describes how you feel about the 
(message). The higher tfie number, |the more 
yoi( think the phrase on the right describi^ it. 
Thejower the number, the more you think the 
: phrase on the left describes it. You could also 
: pick any number in between. Now let's go 
through each set of words. !Please tell me which 
number best describe your reaction to the 
(message). . 



Practical I 2 3 4 5 Not Practical 



Too Short 12 3 4 

Discouraging li 2 3 4 

V Comforting'! 2 3-4 

; W^ll Done 12 3 '4 

Not Informative 1 2 3 4 



5 Too Long 

5 Encouraging 

5 Alarmiiig 

5 Poorly Done 

5 Informative 



6. For Assessing Artwork ■ > 

Just Ipoking at^the drawing (or'i^ib^ur0)f w^^ 
you tfiink it says? ' " v j .^^^ i :^ : 



Is there anything in this dra^rtng (ot pictiiure) ; 
that would bother or off end peojple you know? 




Appendix B 

Estimated Direct 
forFrete Using 
Commeraal Research Firmj 



The following tables indicate approximate costs . 
for conducting pretests with the assistance of 
comnitrcial research /i'rms. Costs wjll vary by ^^on and 
by supplier, so it\;an pay to request bids fronfSeveral 
companies. Also, saving^^l^^ble by- using existing 
facilities and st^ff . HQwever,^i|ikfi any other assessment 
from which jiudgments wilf bb drawn, it is impo):tant 
not to operate on so low a budget that results may be 
compromised. ' \ ■ 

; Fbjpus Group Interviews (estimated for one group of 
ylpijrespondfents) 

\ a;/ Recruitment (general audience) 
Respondents' fees (not always 



i 



necessary) 
c. Facilities, audio taping, 

miscellaneous 
d; Moderator 

.e. Moderator's analysis and report 



y ■ 

Si 



..$a50- 300 

*i00- 150 

75-175 
100- 250 
200 - 400 

$ 625-1,275 



In-depth Interviews (estimaM for 25 interviews) 

a. Ih-hbme interviews $ 500 —1, 125 

b. Clinic or other single isite interviews 250 — 625 
ulation, analysis, and report 250^1,625 

tt. $ 750 -2,^0 



Central Location Intercept (estiihated f<^ 100 ^ 
interviews) • / : ; 

.a; Facilities (e.g., renting space in j 

\ aSshppping mall) $ lOO 

b. ;. R^fuit respondents and conduct * ; 

: inter^e^ys^ • 1^0—2,500 

c. Ta^latipn, analysis, and report 1,000—2,500 

' ^ f^^ $2,050--5,100 

Self "Administered Questionnaires (€l$timated for 20 . 
respondents) , j 

a. Qu^tionnaire and booklet repxp- i / . 
j_ duction and mailing V^ "*. ^ $ 50— 150 ^ 

b. Tabulation, analysis, and te^ort; y 1,50 — 500 

? $ 2P0 - 650 




^AlkpendixC 

?Fretest Reports i^ailable 
;&)m the Office of 
iliatocer Communications 



rrlv • Gdncepte^:f and Radio Messages on V, ; ■ ' / 
■■ • Hazaf ds'of . ^" 

Fdbti!&,attii;iides and perceptions on the prob- 
lem df^^iistos, and concepte 
radio and^ l^ public service announcements, 
^ were assessed through groub sessions with blue 
' collar workers and older men and women. 

• 2. Print Ads on Cancer Myths and Misconceptions 
Public knowledge and prevalenpe of ten 
r myths and misconceptions about cancer were 
■ 5 investigated in a series of eight focus groiip ' 
\ -r. interviews; In addition , three possible formats 
. for print ads about leancer myths were assessed . 

. 3. Brea$t , Cancer Screening 

Focus group in terviewjs were coi^diib^^ 
inner^ity women to generate hy^othe^ for a 
quantitative study of underutilization of a 
breast cancer screening center. Attitudes and. 
beliefs about breast cancer, and use ot fjC^e 
screening or clinic services were assessed. 

-iv.,''' • ■ ■ ' ■■ ■■' .A 
Colon-Rectiim Cancer Detection • 

Awareness, ^ttitudesf and exploratory con- 
cepts were intefiitigated for the American Cancer 
Society /District of Colombia Division, to 
assist in^the development of a public education ^ 
ef f ort on the .detection of colon-rectum cancer. , 




B, Individual 

In-Depth? interviews 



1. 



Adolescent Cancer Patient Concept Papers 
; 'lii^de^th interviews with teenage cancer • 
l)atients'were used to p^.test a serieSH](f 9 
cobceipt papers. The interviews concerni^ the 
relevance of the concept papers to teenag^ 
patieht4> whether any issues i^c^eded to be 
escpand^id, deleted, or added, aiid to obtain ^- r . 
dircK^tioii^or the most appropriate mediuni,(6r 
d^veldpmgmaterials. ^ V' 



bteast);>|r«?^ ampng patients and 

J; tl]!;c^^^^^ the;Co|j^v>:-:^r 
pretiensive CanciB^^ Center of * the State of^^^^.,' / . . 
Florida. Comprehension and reactions to the . ' 
pamphlets and their apprdpriatene^ for 
patients were pretested. - 

' ■ . * ' ■ ■ i' 

3. Breast Cancer Site Pamphlet ' 

Based on the pretest noted above, the breast 
cancer site pamphlet was revised and pretested 'y. 
again in cboperariop with the Northwestern , 
Universit^Meilical. Center's Oncofogy Clinic. 
Paiti^nts and patient^related individuals were 
I interviejyed to assess comprehension of spe- 
cific medic^d tenns compiled in a glossary,an(C^ 
Other reactibnsjtp the tond and content of the 
pamphlet. ; 



EKLC 



40 



89' •■ 




[ V Siiibking Cessation TV Message / f 
Main idea recall arid reactions tp a x\ 
0 vision PSA produced, by the National 
j^-:^^^ oh Smoking and Healtft were 

W^^j^tudiecC^ One htindred interviews withjnen and 
^^ ■^ ^wprner^ smokere and "nonsmokers, Iverei jcon- 
ducted in two shopping malls in New York. 

: 2i>.Bdokte^ How Chemicals Are Tested 

iv^^^: relevjance 
w.eri^re^ for a booklet containing" com- ^ 
piex, technical information iptended fdr the lay. 
public. Central location interviews were con- 
ducted in a suburban New Jersey shopping mall. 
* i> ' . ■ . .• " ■ ■ ■ »* . 

3. Cotfjcepts for a Smoking Cessation Poster 

Miseries of concepts and designs for a smoking 
cesisi^tion poster were tested. Comprehension , 
measures, anil target audience preferences were 
gathered. • 

. \ ■■ ... • 

Melanoma Educatipn Concepts 

Public knowledge, at^itudes/and practices re- 
, garding skin cancer weriCfKvestigated f pr 
I^lassachusetts Genisral Hospital. Al^ :^ual 
concepts and messages fdfir identifyihgiiiQiil 
detecting melanoma )vere tested in centi^^ 
location interviews at construction sites and ' 
^ beachiM ip, the Boston arcja. . 

'^1 Questionnaires 

1. Booklet on Coping witlLCaftcei^ 

Self-administered quratiohnaires were used to 
pretest a long draft manuscript on the psychjp- 
" social issues related to cancer tgt compreheh- 
sion, personal relevance, believability, and other 
target audience perceptions. Resj|pndents also 
^ were told to "copy edit" the manuscript as they 
read,'. ^ > f... 
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2. Booklet for Educators on "Cancer in the 
Classroom** 

Qui^tionnaires and draft booklets were sent ; 
to teachers, principals, counselors, and school 
nurses to gather reactionii tp the i^ooklet, td . 
identify areas that needed improvement, and to 
obtain specific suggestions from educators on ; 
how tP facilitate a young cancer patient's return 
to the classroom. 

3. Booklet on "Testing Chemicals for Cancer" 

Questionnaires and draft booklets weire 
mailed to consumer affairs officers at super- 
market chains, health officers at health depart- 
ments, and doctors. Questionnaires were de- 
signed to gather professionals' judgments of the 
booklet's accuracy, clarity, completeness, and 
appropriateness for lay audiences. . 

E. Health Message ^ 
Testing Service (HMT^^ 

Reports on test resultspCor television and radio 
public service messages that have 4)een~pretested 
through^this s^iCj^ iUre available from the Office of 
Cancer CommUliic^ The subjects of test messages 
include: smoking qessation, breast self-examination, 
maintaining high blood pressure treatment, physical 
fitness, losing weight, preventing drug abuse, testing 
for juvenile hearing loss, using seat belts, asbestos ex- 
posure, healthrrisk appraisal, and promotion of an 
infpirmation hotline. 

To obtain copies of these reports, please contact: 
Rose Mary Romano - 
Office of Cancer Communications 
National Cancer Institute 
Building 31, Room 4B39 

Bethesda,MD 20206 . 
(3(^1)496^792 -y 
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Appendix D 

Glossary 



■ * • .. ■ . 

Attention, A pretesting measure to d|5i84^9be^^ 

ability to attract listener or viewer attcdltio^^ ' 
often called r^all. ^i;;-^:^^^^^^ 



Altitudes. An individual's predisposiiibns towardan 
object, person, or group, which influence his or her re- 
sponse to be either positive or negative, favorable or 
unfavorable, etc. 



Central location intercept-ifiij$nviews .In^ con- 
ducted with respondents wj0^ at a hjghly 
tiaffickdid location that is f^iqii^nted by individuals 
typical of the desired target audience. 

-Closed enSed questions . Questions that provide ire- 
spondents with a list of possible answers from which to 
choose; also called multiple choice questions. 

Communication concepts . Rough art work and'sti^te- 
ments that coftvey the idea for a full ihessagi. ;^ ; : ? 

7 ■• ■ . ■ ' 'jf." 

Communication strategy, A written statement that 
includes program objectives, target audiences, an under- 
standing of the information needs and perceptions of 
each target audience, \^hat actions they should take, 
the reasons why they should act, and the benefits to 
be gained. This document provides the dij^eqtion and 
consistency for all program messages and materials. 

Comprehension . A pretesting mjqa^Ure to determine 
whether .messages are clearly ulitilerstood. 

Convenience samples . Samples that consist of re- 
spondents who are typical of the target audience and 
who are easily accessible; not statistically projectable ^ 
to the entijre population being studied. P'' 

i, ' ■ ' ' • 



Diagnostic information . Results from pretesting re- 
search that indicate the strengths and weaknesses in 
messages and material^. 

Execution . The form or manner in which a message 
strategy is carried out; also called rendition or version. 

Focus group intervie^H.- A type of qualitative re- 
search in which an experienced moderatpt^ jeatls about 
8 to 10 respondents through a discussioh'^oMi Elected 
topic, allowing them to talk freely and spontaneously. 

Formative evaluation. Evaluative research conducted 
during the program development stage; this may in- 
clude state-of-the-art reviews, pretesting messages and 
materials, and pilot (toting a program on a small scale 
before final production!' 

Frequency. Jn advertising, this term is used to describe 
the average number of times an audience is exppsed to 
' Va specific media message. ; * :< 

In-depth interviews . A form of quaHtiative research 
consisting of intensive interviews to find but how 
people think and who^ they feel ab&ut!a given topic. 

Intermediaries organizations, such as profes- 
sional, industrial, ci^c"", social, or fraternal groups, that 
act as channek; for distributing program messag«3;r9^pd 
materials to members of the desired target aiidiehce. 

Open ended questions . Questions that allow an indi- 
vidual to respond freely in his or her own words. 

i Polysyllabic words . Words that contain three or 
more syllables. 
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Pretesting. A type of formative research that involves 
systematically gathering target audience reactions to 
' communication messages and materials before ^hey are - 
ptodiiced in final form. ^ ' 

Process measures. -Evaluation research concerned with 
thie process of program implementation; includes 
assessments of whetheir materials are being distributed , 

.to the 'right people, and in what quantities/ whether 
and to what e3d;ent program activities are occurring, 
and other measures of whether and how well the pro- 
gram is working. 

^Qualitative research. This research is qualitative or sub- 
jective in that it involves obtaining information about - 
feelings and impressions from.small samples of re- 
spondents. The information gathered usually cannot be 
quantified in numerical terms, and hard and iBfX 
generalizations should not be made. 

Quantitative research . Research designed to*g^her 
objective information from representative, random 
samples of respondents; results are expressed in ^ 
numerical terms (e.g., 35 percent are aware of X and 
65 .percent are not). Quantitative data are used to 
make generalizations about the target audience. 

Random sample . A sample of respondents in which « 
every individual of the population has had an equal 
chance of being included in the sample. . 

Reach . In advertising, this' term is used to describe the 
number of different people or households exposed to 
a specific media message during a specific period of 
time. 

Readability testing . Using a formula to predict the 
api^roximate reading grade level a person must have 
achieved in order to understand written material. 

Recall . In pretesting, this term is often used to 
describe the extent to which respondents remember 
seeing or hearing a message that was shown in a com- 
petitive media environment; usually centers on main 
idea or main copy point recall. ^ 



Segmentation i Subdividing an overall population into 
homogeneous subsets of target audiences in order to 
better describe and understand « segment, predict be- 
havior, and formulate tailored messages and programs 
to meet specific needs. Target audiences may be seg- 
mented demo graphically (eg,, age, sex, education, 
family life cycle), geographically (^e.g.. Southeastern 
U.S., rural, north side of town), and by a variety of 
characteristics sometimes called psychogra'phics (e.g., 
personality, lifestyle, usage patterns, risk factors, 
benefits sought). * . , 

Self-administered qi^^sfi<mn(!ires . Questionnaires that 
are filled out by respondents themselves; these can be 
mailed directly to the respondent or they can be filled 
out by groups of respondents who are gathered at a 
central location. 

Summative evaluation. Research designed to sum- 
marize a program's accomplishments and effectiveness; 
also called Outcome or iippact evaluation. 

Target audience. The desired or intended audience for 
program messages and materials (see segmentation). 
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